Papers with global scene understanding
Vision-and-Language Navigation with Analogical Textual Descriptions in LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing zero-shot LLM-based Vision-and-Language Navigation agents either encode images as textual scene descriptions, potentially oversimplifying visual details, or process raw image inputs, which can fail to capture abstract semantics required for high-level reasoning. |
| Approach: | They propose to integrate large language models into embodied AI models by incorporating textual descriptions that facilitate analogical reasoning across images from multiple perspectives. |
| Outcome: | The proposed approach improves the agent’s contextual understanding on the R2R dataset, showing that it can make better decisions based on the LLMs. |
DICA: Dual-Indicator Guided Contrastive Alignment in Multimodal Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Multimodal large language models may deviate from this pattern due to attention drift and underutilization of visual evidence. |
| Approach: | They propose a Dual-Indicator Guided Contrastive Alignment (DICA) that tracks visual attention and output image correlations to improve visual grounding. |
| Outcome: | The proposed model outperforms existing approaches and significantly reduces hallucinations. |